Human-Machine Collaborative Systems for Microsurgical Applications

We describe our current progress in developing Human-Machine Collaborative Systems (HMCSs) for microsurgical applications such as vitreo-retinal eye surgery. Three specific problems considered here are (1) developing of systems tools for describing and implementing an HMCS, (2) segmentation of complex tasks into logical components given sensor traces of a human performing the task, and (3) measuring HMCS performance. Our goal is to integrate these into a full microsurgical workstation with the ability to automatically “parse” traces of user execution into a task model which is then loaded into the execution environment, providing the user with assistance using online recognition of task state. The major contributions of our work to date include an XML task graph modeling framework and execution engine, an algorithm for real-time segmentation of user actions using continuous Hidden Markov Models, and validation techniques for analyzing the performance of HMCSs.